Is OSCE valid for evaluation of the six ACGME general competencies?

Assessment of clinical competencies by using objective structured clinical examination (OSCE) is one of the current mainstreams in medical education. According to the concept of Miller’s pyramid, OSCE is defined as an assessment method belonging to the entity of “shows how”, i.e. an assessment method designed for the evaluation of clinical skills. In this issue of the Journal of the Chinese Medical Association, Yang et al. have developed an OSCE program to assess the six general competencies, including medical knowledge (MK), interpersonal and communication skills (ICS), systems-based practice (SBP), patient care (PC), professionalism, and practice-based learning and improvement (PBLI), advocated by the Accreditation Council for Graduate Medical Education (ACGME). The authors assessed the reliability of the examination and concluded that OSCE could be a reliable method for determining postgraduate year 1 (PGY1) resident’s acquisition of the ACGME core competencies. Besides reliability, a more important and interesting question to be answered is whether or not OSCE is valid for evaluation of the six ACGME general competencies? OSCEwas primarily designed for assessment of clinical skills such as history taking, physical examination, simple procedures, interpretation of laboratory results, patient management problems, communication, attitude etc. According to the ACGME Toolbox of Assessment Methods, OSCE is one of the most desirable methods for evaluation of the relevant skills for PC, ICS, and professionalism; it can also be one of the next best methods for evaluation of PBLI and SBP, but is not considered as a cost-effective method to evaluate MK. The contents of the ACGME Toolbox of Assessment Methods are apparently reasonable. However, the validity evidence of OSCE on assessment of the ACGME general competencies is still limited and necessary for further verification. In a recent study, Berman et al. investigated the validity of the New York City Rheumatology OSCE (ROSCE) on assessment of four of the six ACGME general competencies, i.e., PC, ICS, professionalism, and MK. The authors disclosed promising results; however, the merit of their work is not for providing new concept but solid evidence on the validity of OSCE. As stated by the authors, during the 5-year period of investigation, every attempt on design and evolution of ROSCE is made to create scenarios that mimic typical clinical encounters with realistic patient interactions to improve examination validity. In other words, to build up a valid OSCE